Interactions Between Working Memory, Reinforcement Learning and Effort in Value-Based Choice: A New Paradigm and Selective Deficits in Schizophrenia

RLWM Model RLWM includes two modules, a classic incremental RL module with learning rate a, and a WM module that can learn in a single trial (learning rate 1) but is capacity-limited (with capacity K). The WM module is also subject to forgetting. The final action choice is controlled by weighing the contributions of the RL and WM modules’ policies. How much weight is given to WM relative to RL (the mixture parameter) depends on two factors. First, it depends on what the probability is that a stimulus is stored in WM of capacity K. If there are fewer stimuli than WM can hold (nS<=K), then that probability is 1. Otherwise, only K out of ns can be stored. Second, the overall reliance of WM vs. RL is scaled by factor 0<r<1, with higher values reflecting relative greater confidence in WM function. Thus, the weight given to the WM policy relative to RL policy is w = r x min(1, K/nS).